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ABSTRACT » 

The final audit report (1972) 73) on the 
Education Program of the San Marcos Independent School 
critiques the quality of the project and the audit. The 
the audit are: (1) introductory and general comments on 
of the project evaluation and the comparative findings 
evaluation and the. audit; (2) detailed critique of the 
process, and management evaluation for each component, 
assessment of the instruments used, data collection pro 
analysis techniques, and d^ta anal;^sis presentation; (3 
of the auditor's ofi-site findings and their correlation 
evaluator's data and reports, summairy of consistencies 
discrepancies, and interpretation of discrepancies; (4) 
recommendations for evaluation design revisions with a 
each; and (5) Confirmation or questioning of the need f 
modifications proposed in the pro ject « evaluation. {For 
document, see RC 007 264.) (NC) 
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EDUCATIONAL ACCOMPLISHKEOT AUDIT . 
REPORT FOR SMi I-IARCOS 



Purpose of the Report 
This Audit Report is made in compliance with the contractual agreements, 
legal prescriptions, and official directives under the Provisions of Title VII . 
of Public Law 89-10, as amended, for the este^blishment and operation of .bilingual 
education programs. This is an Audit Report on the Bilingual Education 'Program 
of the/"S^n Marcos Independent School District. - Although this program comprises 
J^"Two~^p\i6lic school districts, San Marcos and Harlandale, and a university. South- 
west Texas State University, separate evaluations and audits are being performed 
for e two school districts this year in accordance with the changed United 
States Office of Education directive. The program is in its fourth year of 
operation, haying as its director this year Mr. Rene Gonzalez. The .purpose 
of 'this report is to give a critique concerning the quality of the project 
evaluation and discuss the comparative findings of the project and the audit, 
and to confinh or question the need for program modifications which have been 
proposed as a result of the project avalua tides. 




r7 Scope of the Report 

. The Audit Repoitt follotJs the suggested Audit Report Content Areas received 
fifom the United States Office of Education which is a 'part of the contractual 
ement between the auditor ^nd the program director as follows; . \ 



1. Introductory and general comments concerning the quality 
of the project evaluation and the comparative findings of 
the project evaluation and -the audit. 
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Detailed critique of the product and process evaluation 
conducted for operation and raanagsment , based on an 
assessment of the instruments used, data collection 
procedures, data analysis techniques, and data analysis 
presentation, 

3, Description of the auditor's on-site visit findings and 
their correlation with the evaluator's data and reports; 
summary of consistencies and discrepancies, and interpretation 
of the discrepancies. 

4, Recomraer\dations for revisions in the evaluation design, in- 
cluding a rationale for* each recotiiraendation. Since the 

^auditor's objectivity can be retained only if the selection 
of a specific corrective action is a local decision, he should 
provide getieral rather than specific recommendations, posing 
several alternative actions or possible sources of assistance 

\ 



to the LEA In correcting the deficiency^. y 



5. Confirmation or questioning of the need for program modifica- 
tions which have been proposed as a result of project evaluation. 

The audit areas described above will be .referred to as sections in this 

report with section numbers corresponding to the audit area description numbers. 

■ r* 

SECTION \ \ y" ' 

Introductory and\^General Comments 
Concerning the Quality and Significance 
of the Final Evaluation Report 

The Evaluation Report for 1972-73 has followed the evaluation design. The 

i ) 

evaluator has accomplished very well the purpose of a bilingual progKim evaluation 



which in this instance has beqn curtailed by federal directives to only the 
instructional component. 

The Evaluation Report" for 1972-73 Is a remarkable .document because of its 

sclent^ic objectivity, penetrating analyses, logical and lucid interpretations, 
per<;.eptive appraisals, and fundamentally sound recommendations. Project and 
administrative officials would not be remiss to .consider the evaluator's findings 
in their future educational plans. 
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This report uses the presentation, analysis, and interpretation of statistical 
data as its basic format. The extensive use of statistical data is an appropriate 
techniqufc> for specific measurement of learning growth and levels of achievement. 
Many of the expected performance objectives are evaluated by means of realistic 
criterion-referenced quantitative measures. At first glance the statistical data * 
may appear to be a bit excessive, however, these data are most appropriate in an 
adequate evaluative* study. 

Concern is often raised by many persons about the relevancy and appropriateness 
of certain evaluation instruments. This concern has been especially strong in the 
testing of minority students. The evaluator of this project has used a wide 
variety of measures vjith several of them administered bilingually or written in 
Spanish and English. The interpretative comments call for appropriate modifica- 
tions, taking into consideration the target pupil population. The diversity in 
individual classrooms and in reasonable achievement expectations is evidenced in 
this report. It is obvious that selection and use of. tests in a project such as 
this need constant review in» search of more valid means of assessment; however, the 
utilization of more accurate methods of scoring these tests and .the availability of 
longitudinal studies lend supporf^^t^ the instruments now being used . The auditor 
has noted that ^ the scores of pupils in the program are being cotrtparcd to their earlier 
performance under the same conditions; therefore, one can argue that the evaluation 
results should hot be criticized, but that learning opportunities need to be assessed 
and replanned. ^Thc evalua.tor has made several constructive suggestions. 

As verified in an interview with th^ evaluator, £he auditor has ascertained 
that the team of examiners as well as the classroom teachers themselves are the 
most experienced assessment staff since the project began four years ago. This ^ 
experienced team has facilitated a more effective individualized oral and written 
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testing program.. These strengths along with electronic computer services ^aid the 
evaluator wit.h more analytical techniques in data interpretation, and certainly 
are commendable additions to the evaluation dimensions. 

The graphic presentations facilitate comparisons and add greatly to the 
comprehension and interpretation of the data by the concerned reader. 'Suggestions 
of remediation aspects for the project are more easily obtained from these visual 
aids. The evaluator has made these graphics explicit. 

The auditor notes that improvements in evaluation techniques usually result 

r ' ; 

in magnification to the visibility of instructional results. This clearer view 
of aspects of the project should not result in dissatisfaction with the evaluation^ 
but on the other hand it sfiould bring into focus certain program aspects in need 
of revision and remediation. 

The objectives of the bilingual project can be summed briefly as: (1) to 
feduce-the educational deficit of the Mexican-American child while instiling in him 

his blcultural heritage: (2) to obtain for the child the advantage of literacy 

j 

in two languages; and (3) to give the Anglo-American child an opportunity to 

• I y 

broaden his outlook by introducing him to a new language and another culture. The 
evaluation report is expected to show some indication of how well these objectives 
hf-ve been achieved this year. An on-going assessment of the project's objectives 
as tp their practicality^^ds to be part of the program itself. ^ 

Objectivity and comprehetis^lveness appear to be major characteristics of this 

report. The recommendfations are educationally sound, clearly stated, and are 

\ 

based on much em-^'.rical evidence. \ 

A Project description, beginning on Pa^ 1 of the evaluation report is an 

accurate account. There are .six objectives of the bilingual program which have 

appropriately been placed as the flrbt part of this section. These intended 

• ' . ' * ^ ^ \ 

outcomes are -the criterion on which the entire project sli^ould be evaluated. Several 



comments through this audit speak to the ^idequacy of the evaluation in obtaining 

4 

these goals. 

Table I, ^'Teachers, Schools,, and Summary of Piipil Data," shows teacher 
assignments by schools, grade levels, and class sizuc. Of the four schools with 
25 teachers in the program, one school involving 10 teachers employs team-teaching 
arrangements. It is commendable that 10% of the 684 studentSj^ those who are not 
Mexican- American, have chosen to become more acquainted with multi-cultural 
education. Although 18X of the children in the program have Spanish-surnames, 
this indicator is becoming more and more difficult to i^se as a device for as- 
certaining instructional needs of children in this locality. 

In spite of the fact that the Consort im, which operates the bilingual . project 
encompases two cities some fifty miles apart, all aspects of the program have 
functioned well. Since each of the two public school districts involved has l^ad 
its own coordinator of the learning component, very little, if any, detrimental 
effect on the instructional program has resulted because of one dictrict being 
fiscal agent, while the university facilities and the other school district were 
in a separate (Jity. 

■ As has been mentioned in the Interim Report, a major change in the program 
this year is the assumption of financing for the second grade as well as the first 
by the local district. This is even greater evidence of the commitment by the 
school district to bilingual education. 

Two major n-^.nagement personnel changes have been made this year — a new 
project director, Mr. Pven/ Gonzalez, and a new coordinator, Mr. Luis Delgadillo. 
The auditor concurs with the evaluator that increased efficiency as well as 
community involvement by these men has enhanced the total bilin^qal project. 

Under the direction of last ye^if's project director, Mr.. Carlos Rodriguez, 



Southwest Texas State University's bilingual-bicultural teacher- training center 
has greatly enhanced this bilingual project. It is easily observed that in 
addition to the pre-service training of teachers, this program has contributed to 
the efficiency of the testing and instructional aspects of the project. 

SECTION II 

Detailed Critique of the Product and Process 
Evaluation Conducted for Operation and t^anagement , 
Based on an Assessment of the Instruments Used, 
Data Collection Procedures, Data Analysis 
Techniques, and Data Analysis 
Presentation 

\ 

It has been observed that this year's federal guidelines curtailed evaluation 
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to the instructional component*, therefore other components, such as staff develop- 
ment, were dealt vzith only incidentally as they affect instruction. Also, directives 
accorded that standardiz'ed tests form the b,ackbone of the evaluation this year and 
other performance objectives haVe been eliminated from the design. It is the 
auditor's belief that a total evaluation of achievement should include various 
subjective assessments as well^ however^ the instruments selected for. this year ' s 
project evaluation are very feasible under the federal directives. 

The auditor's critiqu^and comnnents are limited to his knowledge afid ex- 
periences with the project since the middle of the school year (February 7, 1973), 
The illness of the former auditor, Dr. Guy C. Pryor; necessitated a change. As 
time permitted, every school campus, school principal, and project administrative 
personnel were visited. Thesn visits, test samples, test instruments, as well as 
classroom processes are bases for this report. More informatiorl^ will be related 
later in this report under the seot*ion on **On-Site Visits". 

The project evaluation was* conducted very effectively and\ efficiently on 
schedule as planned and supervised by the evaluator. The kindergarten and first 

ERIC -6- •. 



grade students were evaluated individually on vocabulary by a trained testing 

team composed of Southwest Texas State University's intern student t^eachers. The 

! 

regular classroom teachers administered readiness and achievement tegts in coopera- 
tion with the cvaluator. The auditor has observed that the individual examiners 
a^e quite experienced . due perhaps to the fact that many have worked in the program 
for several years. ^Thp project evaluator can be conincnded f or ^a very highly 
organized and V7ell-planned operation and management of tofcting procedures. This 
function V7as carried out in a most business-like manner. Further comments v/ill 
follow as the auditor critiques the evaluation product and process. I 

Scores for Peabody tests and those for all qther evaluatidn .instruments 

• • ' . • . I " *v 

administered are put on card^ nnu t^lectronicall^ 'procu£;s^d. The process of rcturninf^ 
feedback on these scores to project teachers not only allows for early diagnosing- 

of pupil weaknesses but also facilitates earlj.er corrective action. The auditor 

\ \ . 

observes this as a significant, factor in the learning process-^-ecpecially as the 

I 

number 6f pupils increases, more data is gathered and disseminated in the evaluation 
process. 

• Table J^f;^"Peabody Picture Vocabulary TestSj 'compares 1971-72 dominant 
language means with t.iose of 1972-73' in kindergarten and first grade pupils. 
The significant eviden::e of handicaps in concept formulation presented magnifies 
the need for teachers to implement extensive measures to reduce this deficit. 
The comparison of fall and spring P^abcdy scores reveals in Table III, a much 
lower expectable ga4n in Spanish than in English for kindergarten pupils. Other 

infinitesimal increases in Spanish Peabody scores compared to English are observed 

0 ■ , . . / " ■ ' 

by mean, scores in Tables IV and V. It appedxs^that the recommendations in the 

cvaluati^^i^ report to the coordinator are well founded. The auditor agrees that 

the coordinator should work closely \;ith teachers to -improve the pupil learning 

of concepts in Spanish next yecr in order tp more fully realize the project obV 

\jectives. 



.The Metropolitan Readiness Test and Metropolitan Achievement Test results, 
(Tables VI-VII), indicate a high percentage of success toward behavioral objectives 
for kindergarten and first grade students. The auditor concurs that the bilingual 
pvograrn is successfully fulfilling the objective of reducing educational deficit. 
For those groups who scored very high it is suggested that the evaluator compaxe 
the rates of progress of these pupils succeedin,:^ year^. The graphic presenta- 
tions give a clear picfure of explanations and findings — especially the graphs 
illustrating variability and growth at three quartlle lovels facilitate comparisons, 
(Tables XII--XIII) . These are commendable procedures. 

• The evaluator ireports (p. 25) that in grades 2-5 generally the amount of 
gain Is good for Metropolitan Achievement Test areas of v/qrd discrimination, 
language, math computation, social studies, and occasionally reading coi^rehenslon. 
Percentages in several cf these areas lagged below expectations a great deal in 
the third grade,., however. ' > J 

When qliartile scores were analyzed, (p. 26) a large variability existed 
between classrooms as to the percentage of pupil success. Also, a larg^a vari^- 
bility^exfsted between classrooms as to the area'(s) of pupil success.- The evaluator 
has highlighted the evidence produced by the data that there is a great need for; 
extensive consultation among -the coordinator and teachers. Determining reasons'. . 
,for expertise in some areas and a sharing of "this expertise wj.th f cllowyteachers, 
as WGll'as further college work for needy teachers are suggested. The auditor 
concurs that the coordinator approach this recommendat;;Lon as^ soon as possible iri 
a most professional manner. 

It has becA noted that the pupils fall fur ther -behind the national' norms on [ 
these achievement tests each year that they are in school. The evaluator points 

out (p. 26) that there is no achievement test /on the market at this- time which 

. \ • ■ ' 
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is constructed for or normed for the pupil population represented in this 

\ 

project. It is recommended by the auditor that, whether bilingual projects are 

continued on a locally or federally funded basis, some type of standardized 

achievement assessment device be sought. The school district should keep a constant 

* 

vigil for such an instrument. * 

According to the evaluator, the Inter-Anericana Spanish reading test, the 
Prueba de Lectura,A^as administered to grades two^ three, four, and five. Only 
raw scores were used to compare progress. The test was reported as being 
difficult (p. 26-27), but was the* only one available at the commencement of this 
project and is presently used so that a longitudinal study is possible next year* 
The evaluator is to be commended for the clear and accurate summative analyses 
and comments. Although results are generally good, there is a wide variation 
reported in some mean scores, (Tables XIX-XX) . The coordinator should study the^ 
recommendations of the evaluator so that the level of achievement is acceptable 
in all classroopiG. Perhaps a less difficult reading test might be employed in 
the project after the longitudinal study is made next year. 

- Evaluation V7ith the Bilingual Education Program locally- constructed t^ests 
in social studies and science'were reported as rather good. The coordinator might 
do well to" study Table XXI and attempt to discover reasons for the few exceptions 
not attaining their goals. The statistical ^-test of variance between English 
and Spanish performance, planned next year should prove interesting. The auditor 
recommends that the Bilingual Education Program test ^orms be clearly printed so 

that children will be able to distinguish the pictures without difficulty. 

^^^-^ ^ . 

Tlje data reported in Tables XX^V and XXV, "Inferred Seif-Concept Scales," 
presents creditable evidence thatythe bilingual education program is indeed 
helping many Spanish-surnamed pupils to achieve a more positive self-image. The 



auditor conunends the taachcrs and scorer^ on the amount of time and effort 

ohvious3.y expended in evaluating this phase of the projv^ct. Due to the fact that 

this type of evaluation is highly subjective, the auditor recommends that a 

general meeting of all teachers evaluating, vjith these forms have a brief in-service 

workshop with Dr. Elizabeth McDaniel, author of tha instrument, at the beginning 

* . school ^ ^ 
of next/year. Perhaps a better understanding of hov to score the self-concept 

scale would reassure validity and reliability of this evaluative device. 

The bilingual project Is to be commended in ger4Gral for its attempts to 
prevent drop-out of students. Certainly >' succf?r;s in this endeavor v/ill effect 
the program positively. The two--day conference planned by Mr. Carlos Podriguez 
in November which 45 central and south Texas administrators attended is 
positive approach. ^ 

The evaluator has done commendable work in comparing control groups taught 
in traditional classrooms to those of experimental bilingual groups this year for 
the first time. The comparisons were madfi in second^ third, and fourth' grade 
classes. Severe limitations existed in the study, and realistic assessment reported 
(p. 42) that there seems to bo no advantage indicats^d in either group. Significant 
variables, espeqially • the teacher variable, cou^ not be controlled which contribut'-'.J 
to doubtful validity of the study.. As it appears feasible, more studies of this 
kind are recommended. 

Other components vitally assotiated w.ith the learning process and evaluative 
process but not covered in this report appear to have had a positive direction 
this year. The auditor certainly concurs that the call f or iievot ing of more time 
to concept development and reading comprehension this year was warranted and well- 
founded. The coordinator and supervisors have been vrorkitig diligently toward a 
strong teaching staff. (See p. 44); Parert^x contacts with the schools and their 



interest Is on the up-swing. All of these facts are creditable siv^ns for a 
successful program. 

. SECTION III 

Findings and Observations as a Result of On-Site Visits and 
•Examination of Evaluative Data with a Summary of 
Consistencies and Discrepancies 

The activities engaged in by the auditor on on-site visits included: • 

1. Employed by Mr. Rene' Gonzalez, project director, and Dr. llelene ' 
Harrison, project evaluator/ in early February at which time 
duties of the auditor were discussed. ' ' v ^ 

' • 2. Attended several follow-up conferences with evaluator and 

project director tc. discuss relationships bo:tween evaluating 
and auditing processes. 

3. Examined and approved all tests, rating forms and scales, 
and evaluation instruments. 

4. Interview&i:^t4a^chers and principals. 



5. Observe"^ post-test^administrat ions. 

6. Conferred with testing-team members ^as well as teachers. 

7. Sampled most adequately the test results, opinionnaircs, 
rating s^a^eSj data,'^and^d;^^ processing techniques. 

8. Conferred with the project coordinator, curriculum writter, 
and other resource personnel. • 

<^ 

9. Examined samples of curriculum and instructional materials 
used in the project. . ^ " 

10. Attended meeting of LEA administrators, project personnel, 
elementary principals, and teachers * concerning the implemen- x 

' . ■ tat ion .^f bilingual projects^ 

11. Conferred v/ith LEA administrators concerning plans. 

<^ ' ■ 

12. Visited several bilingual classrooms.^ . > ^ 

13. ' V/ent over the evaluation and audl^t ircpt^rt v;i^+K-evaluator and 

other personnel. • * ' 

The thoroughness of the discussions Let<5^cn the evaluator, the auditor, the 
director, coordinator, and other personnel has " reflected a mininum of discrepancies 



between the evaluation design and findings and the auditor's checking and 

conclusions* .The consistencies are very nunerous indeed. As mentioned in 

Sections I and II, numerous specifics have been icJv_ntified which need not be 

repeated* The many appraisal statements of the evalu-^tion and its findings in 

these sections reflect aspects of the consistencies and discrepancies. The 

completeness of the evaluative clata with its careful -^.nalysis and object j,ve 

interpretation h«?ve reduc .d the discrepancies ? minir.un. TJajor concerns are 

raised relating to the rcconnondat ions of the *evaluator on Page A8 of the report. 

These recommendations are v:ell"f oundcd in the evaluation data. 

•The first recommendation concernin;! minimizing of , pupil drop-out~of-the~ 

progr/im is well founded. Certainly a^'IT^inistrators need be cognizant of 

mairttainlng more pupil continuity. 
f 

The sviggestioq^s. f or tho continuing need for recrui tmentN^f bilingual teachers 
are well establishcfl. 

The recommendations for more effective time allocation in team-teaching 
should be studied and implemen'fced . Concept development in both languages is 
apparently'^Tn need of furthf>r study. 

The .suggestion—^'tlie importanci^ of .spending, less time on phonics, spelling, 
and sound discrimination -and more tim:; on reading comprehension in botlv English 
and Spanish .lani;uage arts.'*-- is strongly indicated by evidence. A limited use of 
cuadcrno^as a, suggestion should be heeded. 

Recommendaltions concurred to by the auditor stated that the coordinators 
•should scrutinize the tallies in the evaluatityi report to determine instances of 
teachers* strengths and weaknesses. A followAip v^ith planned improvement pro- 
ccdures 4i)^/or sharing' of expertise by individual teachers is suggested. 

OnCjC again, tlie auditor agrees unequivocally with these educationally sound 
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recommendations. Thtiy arc clearly stated., completely vnlid, and should be 
implemented in order to m.ake the project the most valid possible. 

V 

SECTION IV 

Recommendations for Revision of 
the Evaluation Design 

Only one minor recommendation for revision within the evaluation design is 
suggested by' the auditor. If a standardized test v;ith regional and/or nat;ional 
norms is available, it should replace tests now bcinp. used vrLth only rav; scores. 
Perhaps this would be more feasible after the project has operated for five years . 
and appropriate longitudinal studies attempted. 

With the scope of the evaluation for this project b»:iinc limited by federal 
directives this year; the auditor recommends no major changes at this time. 
Another look at the uniformity of scoring by various teachers on the pupil self- 
concept scale might be considered. 

SECTION V 

Confirmation or Questioning of the Need for Program 

Mod^^cations which Have Been Proposed as r^^^^ 

A Result of Project Evaluation ^ 

Tlic auditor of this project has found no cause to'^'^uest^n the recommendations 
for program revisions. A strong endorsement of thcsQ suggestions is appropriate. 

The classroom teacher is the most important facitor effecting the outcomes 
of the bilingual, bicultural educational projects The auditor most sincerely 
confirms that any recommendations for performance improvements of the classroom 
teacher should be re-emphasized. If the behavioral objectives of the program are : 
to be achieved in a more effective manner, the teacher must constantly plan proper 
activities which will give children opportunities to practice the intended terminal 
behaviors. 
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It is most proper that thtj suggested modifications based on revelations of 
the project evaluation be implementeid with all haste. 

Summary of Recommendations * 
and Comments 

. ■ • 

1. The Final Evaluation Report for 1972-73 has been limited in' 
scope as compared to former years, yet is very comprehensive 

' and most specific, 

2. The evaluation is accurate, and the data have been properly 
analyzed and interpreted. 

3. The recommendations and suggestions ^made by the evaluator are 
derived from thorough assessments and, are educationally sound and 

- administratively feasible. They should be studied, by all educators 
associated with ^ the project and th^n implemented. 

A. The LEA administrators should be c/gnizant of a continuing need 
for pupil retention in the project^, jas well as a-cohstarft need for 
bilingual teacher recruitment 

5. Although there is a continued effort for improving the learning 
of* concepts in both English and Spanishj a more concerted effort 
should be made. Also, Spanish-language teaching should be given 
more time, especially on lower grade levels. 

— 1- _ '/ . 

*6. According to suggestions, more tim? should be spent on reading' 

comprehension in both English and Spanish and less time on phonics, 
spelling, and sound discrimination. 

7. As has been stated earlier, a careful look at tables in the project 
evaluation should be made to determine weak and strong areas for 
respective teachers. ^Remedial or refresher measures and/or a 

• ' sharing of expertise should follow for and by these teachers. 

8. The administrators of San Marcos Independent School District and ' 
^Southwest Texas State University along with the^ Harlandale Ihde- 

4 ' pendent School District are to be commended for having initiated 

and made provisions for a bilingual instructional program through 
these four years with improvements each year. 

9* Th*e auditor has observed that the evaluator has been very responsive 
to suggestions, and discussed these in a professional manner. Each 
recommendation has been responded to promptly with appropriate and 
satisfactory dispatch. The, Evaluation Report is highly commended. . 

10. the project has much merit and should be funded in some manner 

after thc*^ projected five-year period. ' ' 

/.-■' 
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